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Abstract 

We compare two different definitions for the wavelet entropy associated to stochastic 
processes. The first one, the Normahzed Total Wavelet Entropy (NTWS) family 
[Phys. Rev. E 57 (1998) 932; J. Neuroscience Method 105 (2001) 65; Physica A 
(2005) in press] and a second introduced by Tavares and Lucena [Physica A 357 
(2005) 71]. In order to understand their advantages and disadvantages, exact results 
obtained for fractional Gaussian noise (— 1 < a < 1) and the fractional Brownian 
motion (1 < a < 3) are assessed. We find out that NTWS family performs better 
as a characterization method for these stochastic processes. 
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1 Introduction 



The advantages of projecting an arbitrary continuous stochastic process in 
a discrete wavelet space are widely known. The wavelet time-frequency rep- 
resentation does not make any assumptions about signal stationarity and is 
capable of detecting dynamic changes due to its localization properties [H, 0]. 
Unlike the harmonic base functions of the Fourier analysis, which are precisely 
localized in frequency but infinitely extend in time, wavelets are well local- 
ized in both time and frequency. Moreover, the computational time is signifi- 
cantly shorter since the algorithm involves the use of fast wavelet transform in 
a multiresolution framework ['^. Finally, contaminating noises' contributions 
can be easily eliminated when they are concentrated in some frequency bands. 
These important reasons justify the introduction, within this special space, of 
entropy-based algorithms in order to quantify the degree of order- disorder as- 
sociated with a multi-frequency signal response. With the entropy estimated 
via the wavelet transform, the time evolution of frequency patterns can be 
followed with an optimal time-frequency resolution. In this paper we focus 
on two definitions for this quantifier: the Normalized Total Wavelet Entropy 
(NTWS) family introduced recently by us jEBSI^, and another definition 
given by Tavares and Lucena 01 • We compare their performance while charac- 
terizing two important stochastic processes: the fractional Brownian motion 
(fBm) and the fractional Gaussian noise (fGn). In particular, we will show 
that the NTWS family gives a better characterization for both of them. 



2 Wavelet quantifiers 



2.1 Wavelet energies 



The Wavelet Analysis is one of the most useful tools when dealing with data 
samples. Any signal can be decomposed by using a wavelet dyadic discrete 
family {2^^'^^(2H — k)}, with j,k E 1^ (the set of integers) — an orthonormal 
basis for L^(]R) consisting of finite-energy signals — of translations and scaling 
functions based on a function il): the mother wavelet 0,1^. In the following, 
given a stochastic process s{t) its associated signal is assumed to be given 
by the sampled values S = {s{n),n = 1, ■ ■ ■ ,M}. Its wavelet expansion has 
associated wavelet coefficients given by 

C,{k) = {S,2^/'^{2^ --k)), (1) 

with j = —N, ■ ■ ■ , —1, and = log2 M. The number of coefficients at each 
resolution level is Nj = 2^M. Note that this correlation gives information on 
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the signal at scale and time 2~^k. The set of wavelet coefficients at level 
j, {Cj{k)}k, is also a stochastic process where k represents the discrete time 
variable. It provides a direct estimation of local energies at different scales. 
Inspired by the Fourier analysis we define the energy at resolution level j by 

s,=j:^\c,{k)\\ (2) 

k 

where E stands for the average using some, at first, unknown probability 
distribution. In the case the set {Cj{k)}k is proved to be a stationary process 
the previous equation reads 

£, = N,E\C,ik)\\ (3) 

Observe that the energy Sj is only a function of the resolution level. Also, 
under the same assumptions, the temporal average energy at level j is given 
by 

£j = ^X^\C,ik)\' = K\C,{k)\\ (4) 

i k 

where we have used eq. (3) to arrive to the last step in this equation. Since we 
are using dyadic discrete wavelets the number of coefficients decreases over the 
low frequency bands (at resolution level j the number is halved with respect 
to the previous one j + 1); thus, the latter energy definition reinforce the 
contribution of these low frequency bands. 

Summing over all the available wavelets levels j we obtain the corresponding 
total energies: £^totai = 'Ejl-N^j and ^totai = 'Ejl-N^j- Finally, we define the 
relative wavelet energy 

= (5) 
and the relative temporal average wavelet energy 

p, = (6) 

Both supply information about the relative energy associated with the different 
frequency bands. So, they enable us to learn about their corresponding degree 
of importance. Clearly, J^JI^mPj = J2jl-NPj = Ij both define probability 
distributions: {pj} and {pj} — they can also be considered as time-scale energy 
densities. 



2.2 Normalized Total Wavelet Entropy family 



The Shannon entropy provides a measure of the information of any dis- 
tribution. Consequently, we have previously defined the family of Normalized 
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Total Wavelet Entropy (NTWS) as 0, 



-1 

Sw{N) = - ^ Pj ■ logaPj/S'max, (7) 

j=-N 

and, 

_ -1 

S'w(iV) = - XI Pi ■ log2Pi/'S'max, (8) 
j=-Af 

with S'max = log2 N. It has been adopted the base-2 logarithm for the entropy 
definition to take advantage of the dyadic nature of the wavelet expansion; 
thus, simplifying the entropy formulae that will be used in this work. 

2.3 Tavares-Lucena Wavelet Entropy 

Alternatively, Tavares and Lucena, following the basis entropy cost concept 0], 
have recently defined another probability distribution: 

p,, = m\C,{kt /£[];^^ and p^ = E|(5,0)|V4?\ (9) 

where is the scaling function having the properties of a smoothing kernel (see 
reference for details), and ^^T^^ = E |Cj(A;)|^ + E | (5, 0) |^ Therefore, 
they propose the following entropy 

5(j^)(iv) = -( x: 'Y^p,k\og,p,u+p^\og,pA/s^:i^i (10) 

\j=-N+l k=0 ) 

with 5^^^) ^ log2(2^ - 1). As a matter of comparison we have normalized 
this expression and it will be referred as Tavares-Lucena Wavelet Entropy 
(TLWS). 

It should be noted that in eqs. (7), (8), and (10) the maximum resolution 
level is an experimental parameter. It appears explicitly as a direct conse- 
quence of sampling. Tavares and Lucena underlined this fact because it is not 
mentioned in previous approaches. 



3 Theoretical results and comparison 

The aim of this paper is to study the performance of the wavelet entropy def- 
initions previously given. So we a naly ze two well known stochastic processes, 
namely, the fBm and the f Gn B ^3 • The energy per resolution level j and 
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sampled time k has been already evaluated for the fBm 0, 2| . But it can 
be extended to fGn — see the Appendix. The final form reads 



E 



2 c]j 2"^" 



where — 1 < a < 3 — by continuity we have added a = 1 but it does not belong 
to any existent process. It should be noted that the latter is independent of 
k. In the following we will use this power-law behavior with different ranges 
for a, for the two stochastic processes under analysis, gathering both into a 
unified framework. According to its values, the coefficient a must be attached 
to one of the two mentioned processes. 

In order to calculate the NTWS family, the relative wavelet energy for a finite 
data sample is obtained from eqs. (5) and (11) 



Pj 



2-{i+i)("-i). 



1 - 2 



Q-l 



2Af(a-l) 



;i2) 



Similarly, the relative temporal average wavelet energy — see eqs. (6) and (11) — 
gives 

h = 2-^^-"^^"3^. (13) 

Consequently, the normalized total wavelet entropies can be easily obtained 
from eqs. (7) and (8), 



S'w(A^, a) 



logs AT 



1 



N 



2-{a-l) 



1 - 2-^("-i) 

1 



logs AT 



loga 



1 _ 2("-i) 
1 _ 2^("-i) 



(14) 



and 



Sw{N, a) 



a 



log^N 



.1 - 2-" 



N - 

_ 2-^0 



logs AT 



logs 



1-2° 
1 - 2^^ 



(15) 



For the Tavares and Lucena's approach similar steps should be followed. From 
the power-law behavior mentioned before a straightforward calculation yields 
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Therefore, the TLWS is obtained replacing the above into eq. (10), 




log2(2^ - 1) Ll - 2-("+i) 1 - 2-^("+i) 



1 



logs 



1 _ 2(°+^) 



(17) 



log2(2^ - 1) 



1 _ 2-^("+i) 



The NTWS family and the TLWS, as a function of a and A'^, are depicted in 
Figs. 1 to 3. One point to emphasize from these graphs when a > is that the 
NTWS's range of variation increases smoothly with N, improving detection; 
on the opposite, the TLWS's range decreases when N increases. All entropies 
equally improve with N on the — 1 < a < branch. Moreover, for any N the 
NTWS family covers almost all the available range between and 1, while the 
TLWS roughly covers a 25% of this range. 

It is of common understanding that high entropy values are associated to a 
signal generated by a totally disordered random process, and low values to an 
ordered or partially ordered process. If the process is noisy, its signal wavelet 
decomposition is expected to have significant contributions to the total wavelet 
energy coming from all frequency bands. Moreover, one could expect that all 
the contributions being of the same order. Consequently, its relative energies 
will be almost equal at all resolution levels and acquire the entropy maximum 
value. While a nearly ordered process will have a relative energy contribution 
concentrated around some level j, thus its entropy will take a low value. The 
only entropy in concordance with this intuitive vision is Sw, depicted in Fig. 2. 

In Fig. 4 we compare the two entropy formulations as functions of the a- 
parameter when A?^ = 12. It is clear that the (Sw and S^^^ entropies attain 
their maxima at a = (white noise), and the (S'w entropy reaches it when 
q; — > 1. There are two different regions to examine: 

• fractional Brownian motion, 1 < a < 3; 

All the three quantifiers have their maximum at a = 1, and monotonically 
decrease to find their minimum in a near regular process, a ^ 3. The range 
of variation of the TLWS is AS^^ = 0.038, and the range of variation of the 
NTWS family is A^w = 0.384 and A^w = 0.698. Clearly, due to the small 
range of variation, the TLWS is unfit to differentiate between the short- and 
long-memory fBm family members, 1 < o; < 2 and 2 < a < 3 respectively. 
The NTWS family seems to be the best tool for this differentiation, and the 
S'w has the best performance in this interval. 

• fractional Gaussian noise, —l<a<l: 

The TLWS seems inadequate to describe this range — note that 5'^^-' (12, — 1) < 
SS''\12,3). The yS'w is the best suited to describe these noises, since it is 
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monotonically decreasing and presents a range of variation A5'w = 0.698. 
While the confuses between noises coming from short- or long-memory 
processes, — 1 < o; < and < a < 1 respectively. It has its maximum at 
a — (white noise). 



4 Conclusions 



We have introduced exact theoretical expressions for the wavelet entropies 
associated to fGn, — 1 < a < 1. In particular, the range — 1 < o; < 0, to our 
knowledge, has never been studied. 

We have shown that, at least to characterize fBm's and fGn's processes, the 
NTWS family seems to be a better quantifier than TLWS. In particular, the 
fulfils all the requirements for a correct description of the overall a-range: 
has its maximum at the white noise, differentiates between noises and pro- 
cesses, and has the maximum range of variation, AS'w = 0.827. Nevertheless, 
the S'w is best suited to discern between different fBm processes. Finally, in 
the a > case, an inverse dependence on N is observed: the NTWS family 
increases its performance as N increases and the TLWS improves its perfor- 
mance as N decreases. Although the NTWS family always improves with N 
for any a value. 

The procedure outlined in Sec. 2.1 can be followed to build new probability 
distributions associated to wavelet resolution levels. The weight of each res- 
olution level could be modified according to the requirements of the physical 
problem under study. 
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APPENDIX 



Following the methodology described in Perez et al. let us take as the 
signal the noise s{t) = W^{t,uj) — u is fixed and represents one element of 
the statistic ensemble and it will be omitted hereafter. Now, using the chaos 
expansion described in Ref. any fractional Gaussian noise can be written 
as 

oo 

w''it) = J2MHUt)n,„{^), (18) 

n=l 

where {^n}neN are the Hermite functions, and the operator Mh is defined as 
follows 

M^{u) = CH\iy\'^^-''$iiy), (19) 

where the hat stands for the Fourier transform, cjj = r{2H + 1) sin(7ri7), and 
(j) is any function in L^(]R). 

Given the orthonormal wavelet basis {2^/'^ip(2^ ■ — A;)}j,fcez = {i'j,k}j,kew., we 
obtain 

oo 

Cf"(A;) = (W^^,^,,,) = Y.{MH^n,i^,,k)n,A^). (20) 

n=l 

Now we are free to work with the individual coefficients 

d^{j,k) = {MH^n,^j,k) =CH f \u\'/'~''U^)i^,,kMdu. (21) 



Since, the Fourier transforms of the Hermite functions and the wavelet are 
^n{^) = i^~"'^n{^) and 'ipj^kW) = 2~-^ exp(— z2~-^fcz/)'^(2~V), respectively. The 
evaluation of the coefficients d^{j, k) is straightforward from their definition: 

dn{j,k) = c^^i-"2-(^-i/2)^- / \u\'/'~''^{u)U2''y)e-""'du, (22) 

JR 

where ^E'(z^) = 



The chaos expansion in eq. (18) corresponds to a Gaussian process 1J|, then 



under the same procedure used in Ref. Ill the mean of the squared coefficients 
results 

E C^"{k) ' = 42-^(2^^-1) f |^|-{2/^-l) |vl;(^)|2 

Jr 

= 2Ti2H + 1) sin(7rif)2-^(2//-i) / ^-{^h-i) i^. n|2 ^23) 
for any ^ decaying fast enough. 

In the case of the fractional Gaussian noises a = 2H — 1, as opposite to 
the fractional Brownian motion where a = 2H + 1. For the latter we have 
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previously reported ^ that 



E 



Cf^'ik) ' = 2T(2H + 1) sin(7r/7)2-^(2^+i) H iy-^^^+^^ |^(z/)|' du, (24) 

JO 



for any mother wavelet satisfying Jj^ip = 0. Therefore, these two expresions, 
eqs. (23) and (24), can be combined in one written in terms of the power a: 



E 



cm 



242-^" / z/-" |^(z/)P dz/, 



(25) 



where — l<Q;<lorl<a;<3, and Ch is calculated from the value of a. 
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alpha 

Fig. 1. NTWS entropy Syj as a function of a and N. 




alpha 

Fig. 2. NTWS entropy Sw as a function of a and A^. 
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Fig. 3. TLWS entropy S^^^ function of a and A^. 




a 



Fig. 4. NTWS and TLWS as functions of a with N = 12. 



12 



